Semantic Video Indexing Using a Probabilistic Framework

This paper proposes a probabilistic framework for semantic video indexing. The components of the framework are multijects and multinets. Multijects are probabilistic multimedia objects [6] representing semantic features or concepts. A multinet is a probabilistic network of multijects which accounts for the interaction between concepts. The main contribution of this paper is the application of a graphical probabilistic framework to build the multinet. The multinet enhances the detection performance of individual multijects, provides a unified framework for integrating multiple modalities and supports inference of unobservable concepts based on their relation with observable concepts. We develop multijects for detecting sites (locations) in video and integrate the multijects using multinet in the form of a Bayesian network. Detection performance is significantly improved using the multinet.